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I. INTRODUCTION 



A growing interest in measuring success in higher education has emerged in the last few years. 
This Interest, largely fueled by scarce resources and the need to make choices between funding 
alternatives, has resulted in many proposed plans for measuring community college outcomes 
and judging their success, 

A definition of a "successful" community college is somewhat dependent on the perception of 
the person judging the success. Success from the legislative viewpoint might be measured in 
terms of number of students receiving the AA degree and the cost for each student. Success to a 
university administrator may be the number and quality of transfer students received from 
the college. Success to the student may be measured in terms of whether he/she could take the 
two courses needed for a promotional opportunity at work. Therefore, the d&lnition of success 
is somewhat dependent on the set of experiences and framework used to judge that success. It is 
a highly subjective rating dependent on needs and experiences. 

Even if constituencies could agree on the criteria for success, there are so many variables 
involved that it is difficult to isolate what makes one college successful as compared to 
another. However, people do make comparisons between colleges and have perceived 
differences related to quality. Many times funding and/or program decisions are influenced by 
these perceptions. 

In the past, the main technique used to distinguish successful colleges has been a "panel of 
experts" approach. fTiese studies pick successes and then by to determine what they have in 
common.) This technique uses experts to rate and collectively determine successful colleges. 
Critics of this approach cite the usually small number of experts and failure to predetermine a 
broad-based group of indicators of success as the major shortcomings of this method. 

To address some of these concerns, the k^search Commission of the California Association of 
Community Colleges (CACC) designed a study to query a variety of constituency groups 
regarding their perceptions of community college success. The Commission wanted to 
approach the issue of defining college effectiveness in a more quantitative way. Therefore, it 
was proposed that CACC conference attendees composed of students, faculty, administrators 
and trustees be surveyed to determine their perception of the relative importance of a large list 
of potential indicators of college effectiveness. 



II. METHOD 

To determine a comprehensive list of indicators, the Commission members all contributed 
indicators which they thought were important during a Commission meeting in Spring, 1987. 
That list was compiled and sent to each Commission member for further comment and 
refinement. The final list was discussed at length at a second Commission meeting. The major 
discussion centered on whether to limit the list of indicators to include only outcome 
indicators such as: "Number of Students Receiving Degrees," 'Vocational Program Completion 
Rate," or to also include process indicators such as: "Financial Stability' and "Positive 
Reaction of Staff to Students." The Commission decided to include both outcome and process 
indicators. 

As the list of indicators was developed, the study team realized that a variety of measures could 
be used to evaluate how a college was performing on any given indie? tor. The measures selected 
for each indicator could affect the racking of the indicator. Therefore, the study was designed 
to: 1) identify indicators and 2) identify appropriate measures for each indicator. Since the 
task of selecting appropriate measures was more complex than the identification of indicators 
and required some knowledge of research methods, the study team decided to test and refine 
this part of the study at the California Association of Community Colleges Research 
Conference in April, 1988. 



The Indicator study was conducted at the 1987 CACC Convention during two general sessions. 
The first round was distributed at the opening session, and the second round at a Saturday 
luncheon. In the first round, convention participants % were given a survey form (Appendix A) 
which asked them to rank, on a four-point Llkert-type scale, the Importance of 33 "Indicators 
of Successful Community Colleges." Respondent demographic information was also collected 
which included identification of respondent's constituency group, age and location, type and 
size of district. 

The results of this first survey were tabulated during the conference and reported at the 
Saturday general session. The data were analyzed by each group: students; faculty; college 
administrators; district administrators; <:nd trustees. The indicators were ranked according 
to overall mean score. 

The top 19 indicators were selected for the Round Two survey. Nineteen indicators were chosen 
because there was a substantial dilFerence in the mean score between the 19th and 20th ranked 
indicator. In the second round, the respondents were asked to rank order the indicators from 
1-10 with number one as the most important indicator. Round Ti^o was conducted after the 
results were summarized from the first round (Appendix B). 

The second part of the study was conducted at the Research Conference in April. One general 
session was devoted to selecting measures for the top ten indicators. Approximately 60 
conference participants were provided with the ten indicators and examples of measures for 
each indicator. To select the measure, a discussion group was formed for each indicator, and 
participants self-selected into one of the groups. Most groups had at least six participants. 
Each group discussed which measures were appropriate for its indicator. The group developed a 
list of measures it felt were suitable. From these lists a survey was developed. The survey was 
distributed to all conference participants, asking them to rank the top four measures for each 
of the ten indicators. (Appendix C) 



III. INDICATORS OF EFFECTIVENESS 

Tables I through X contain the results of the survey conducted at the 1987/88 CACC 
Convention. The following questions and answers highlight the major findings: 

1. Who responded to the questionnaires on indicators? 

Respondents to Round One and Round Two were quite similar. The smallest group 
represented was Students (5-7%), followed by Trustees (15-19%). Faculty (30-33%) and 
Administrators (44-47%). (See Table 1) 

2. What indicators received the highest rankings from conference participants? 

The top three indicators in both Round One and Two of the survey all relate to faculty/ 
student interaction. The highest mean ranking was "Faculty Effectiveness ' (3.82), followed by 
"Positive Faculty/Student Relationships" (3.68) and "Student Satisfaction with Quality of 
Instruction." Other indicators ranked in top five of Round One or Two are: "Financial 
Stability," "Effectiveness of Administrative Staff 1 and "Positive Reaction of Staff to Students." 
All indicators were considered to be at least "Somewhat Important." There was a spread of only 
.97 on a 4.0 scale between the highest indicator and the lowest. (See Tables n and HI) 

3. Were outcome measures and process measures selected equally? 

Process measures ranked higher than outcome measures. Only three outcome measures 
appeared in the top 19 in Round One: "Student Satisfaction with Quality of Instruction;" 
"Student Satisfaction with Services;" and "Retention of Students." (See Table II) 
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4. How consistent were the rankings between the respondent groups? 



All groups, except Students, rated "Faculty Effectiveness" as the most important 
Indicator. There was much more consensus on the most Important Indicator than there was on 
the least Important. This Is true throughout the study. .As was expected, as the list of Indicators 
was narrowed from Round One to Round Two, the groups showed more consensus in their 
rankings. During Round Two, students also ranked "Faculty Effectiveness" as the most 
important Indicator. This greater consistency may be due to the fact that respondents v re 
given a list of Round One rankings, and these rankings were reviewed before Round Two 
surveys were completed. 

5. How consistent were the rankings between Round One and Round Two? 

If respondents in Round Two had been influenced solely by Round One results, then Round 
Two rankings should have stayed trie same. This did not occur. The largest shifts occurred in 
the following indicators: #5, "Effectiveness of Administrative Staff." had a Round One rank of 
9 and a Round Two rank of 5; "Effectiveness of Planning" went from a rank of 16 up to 1 1; 
"Ongoing Program Development" went from a rank of 10 down to 14; "Effectiveness of District 
Governing Board" went from a rank of 19 up to 15; "Ability to Attract New Students" went from 
a rank of 12 down to 17; and "Sufficient Equipment to Support College Programs" from a rank 
of 14 down to 18. (See Tables IV and V) 

6. Did each respondent group rank Indicators the same from Round One to Round Two? 

The respondents were quite consistent in their rankings from Round One to Round Two. 
In selecting the top ten indicators, District Administrators and Students selected the same 9 
indicators in both Round One and Two; College Administrators chose the same eight; and 
Faculty and Trustees selected the same seven indicators in Round One and Two. 

7. How did the respondent groups differ in their ranking of the indicators? 

Some indicators were more important to certain respondents 1 groups than others. A few 
observations include: 

1) "Effectiveness of Administrative Staff 1 was ranked fifth by College/District 
Administrators and Trustees, but Faculty ranked this indicator eighth, and Students 
fourteenth. 

2) "Community Perception of College" was ranked much higher by College/District 
Administrators and Trustees than by Students and Faculty. 

3) "Adequate Facilities" are ranked higher by Students bad Faculty. 

4) "Effectiveness of Governing Board" was in the top ten only for District 
Administrators and Trustees, and was ranked last by Faculty. 

5) Only Students ranked "Ability to Attract Students" in top nineteen. 

6) Trustees were the only group who did not rank "Student Satisfaction with 
Services" in the top ten. 

7) "Effectiveness of Articulation" is ranked lowest by three of the five groups. 

8) "Student Satisfaction with Quality of Instruction" appears to be of lower 
value to Trustees than with arty other respondent group. 

9) "Financial Stability" ranged from a low percentage of 52.6% for students to 
a high of 90.6% for Trustees. 

(See Table VI and VII) 
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10) "Effectiveness of Administration'' similarly ranged from a low of 47.4% for 
students to a high of 75% for District Administrators. 

1 1) "Ongoing Program Development" was chosen by 63% of the Students, but 
only 33% of the College Administrators. 

12) Only one-half or less of students and faculty felt that "Community Perception 
of College" should be ranked in top ten, whereas close to 70% of the Administrators 
ranked this indicator in top ten. 

8. Do respondents from Northern and Southern California agree on the top ten indicators? 

The North and South agree on the top four indicators. The other rankings appear closely 
aligned with a few exceptions. 

"Student Satisfaction with Services" appears to be relatively more important to Southern 
colleges and "Facilities" more important to the North. "Community Perception of the College" 
appeared in top ten of the North, but not the South. "Effectiveness of District Governing Boani" 
appears in the South list, but not in the North. (See Table VIII) 

9. Do respondents' ages affect their ranking of the top indicators? 

Age was used to categorize responses to each of the top nineteen indicators. Little 
variation occurred in the top four indicators. Much variation occurred in the following: 

Valued Mor e Highly hv Those Under 50 Valued More Highly bv Those Over 50 

Student Satisfaction with Services Adequate Facilities to Support Progress 

Effectiveness of Planning Positive Management/Certificated and 



(See Table IX) 

10. Do employees from single campuses and multi-campuses respond in the same way? 

Another respondent group considered in the analysis was single campus employees as 
compared to multi-campus employees. Very few differences occur. Those exceptions are as 
shown below: 

Valued More Highly bv Single Camnus Valued More Highly bv Multi-campus 

Student Retention Positive Reaction of Staff to Students 

Comprehensiveness of Curriculum Positive Management Certificated 



Classified Relationships 



Ongoing Program Development 



Effectiveness of District Governing 
Board 



Ability to Attract New Students 



Effectiveness of Administrative Staff 



and Classified Relationships 



Effectiveness of District Governing 
Board 



Effectiveness of Articulation with 
Other Post-Secondary Institutions 



(See Table X) 
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IV. MEASURES FOR TOP TEN INDICATORS 

The second part of the study conducted at the Asilomar Research Conference produced a 
set of measures for each of the ten Indicators. Hie following is a list of indicators and their 
measures: 

#1 ~ FACULTY EFFECTIVENESS 

A. Retention rates from 1st census to 2nd census to end of term. 

B. Degree to which students meet stated course objectives. 

C. Student perceptions of faculty effectiveness. 

#2 - STUDENT SATISFA CTION WITH QTTATJTY OF INSTRUCTION 

A. Currently enrolled students' perception of instructors from required, 
standardized evaluation form. 

a Students 1 self-reports upon exiting a course. 

C. Students' seli-reports gathered after leaving (with a time interval, e.g.: 
one semester later). 

#3 ~ POSITIVE FACULTY/STUDENT RELATIONSHIPS 

A. Students' perceptions of faculty as positive role models. 

B. Did student feel recognized as an individual in this class? 

C. Does instructor give adequate feedback regarding student progress? 
#4 - FINANCIAL VIABILITY* 

A. Proportion of expenditures used from reserves for college operation. 
Percent of general reserves to annual budget. 

C. Expenditures per unit of workload. 

D. Extent of financial planning. 

#5 ~ EFFECTIVENESS OF ADMINISTRATIVE STAFF 

A. Ability to motivate college community. 

B. Staff morale. 

C Evidence of key decisions being linked to planning process. 
D. The extent faculty /staff are involved in decision-making process. 
#6 - STUDENT SATISFACTION WITH SERVICES (what a college provides outside classroom) 

A. Satisfaction survey while in attendance (a student at the college). 

B. Satisfaction survey after leaving (after separation from college). 

C. Satisfaction survey at time of service delivery. 

D. Student interviews assessing their experience. 
•All measures should be assessed over ejie (5 years). 



ERIC 



#7 POSITIVE REACTION OF STAFF TO STUDENTS 

A. Perceptions of students assessing helpfulness of college staff. 

B. Staff satisfaction with job environment. 

C. Frequency with which students make use of faculty office hours. 

D. Instructors' attitudes of student learning. 

#8 - COMMUNITY PERCEP TION OF THE COLLEGE 

A. Solicited community attitudes using various methods (written surveys, telephone 
contacts, focused interviews) of various groups such as employers, high school 
counselors, under-represented populations. Formal needs assessment including 
perceived attitudes is the most comprehensive method. 

E. Percent of local high school graduates attending college. 

C. Participation rate of population of the District in college activities. 

D. Number of requests for service from business, industry and other agencies. 
#9 ~ ADEQUATE FACILITIES TO SUPPORT COLLEGE PROGRAMS 

A. Number of hours of room availability compared number of hours used. 

B. Flexibility, adaptability of floor space to meet changing needs. 

C. Adequacy of equipment to meet current instructional needs. 
#10 - RKTEN TION OF STUDENTS 

A. Total units completed, divided by total units attempted at first census (unit 
earned rate). 

B. Re- enrollment rates *over time). 
(Appendix 0) 



V. POLICY IMPLICATIONS AND RECOMMENDATIONS 

Measuring institutional effectiveness Is a complex task, and defining effectiveness Is one 
of the first steps to be addressed In accomplishing that task. The indicator and measures list 
developed in this study my be used as a resource in a study of institutional effectiveness. 
However, users should consider the following: 

1. An indicator is defined as something "that points out. gives an indication of. or expresses 
briefly or generally." Indicators arc not meant to be precise measurements. They may show a 
tendency or a direction when a number of them are studied together. Care should, therefore, be 
taken to ^ ^nble and look at indicators in a general sense and not use them specifically to 
measure success. 



2. Althou'ih there Is consistency among the different constituency groups in ranking the top 
four indicators, variation? occur between groups for Indicators below the top four. Perhaps 
respondents have well-deiued ideas regarding what the niost important indicators are, 
because these (issues have been discussed before. Conversely* they may be less concerned about 
precisely rating indicators of lesser importance. ConsJ' *ency improved somewhat as the list 
was narrowed from nineteen to ten indicators. However, there were still some very different 
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perceptions about the Importance of Indicators. The differing perceptions of what constitutes 
success would have to be addressed before any study results would be acceptable to all groups. It 
might be of some benefit to assemble a representative group and try* to come to a consensus 
regarding some of the discrepancies. 

3. This study involved only internal constituency groups, those persons directly involved in 
the process of community college education. Other groups of individuals outside the actual 
process have a great deal of influence on the process. It would be of benefit to query legislators. 
Board of Governor members and other groups who have direct impact and interest in the 
success of community colleges to determine what indicators would be important to them. 

4. While this study made progress in defining the indicators and measures, which were 
perceived to be appropriate, it stopped short in determining procedures to be used to obtain the 
information. In order for colleges to assess their effectiveness using the proposed indicators 
and measures, procedures might be assembled from the existing literature or new materials 
created as appropriate. 

d. The initial list of indicators included both process and outcome indicators. The 
researchers were somewhat surprised when certain outcome indicators failed to be selected in 
the top ten indicators. One reason for this may be that the professionals in the field realize 
that the procedures used to measure these outcomes can be very complex. Community college 
personnel may be concerned that a simplistic approach would provide distorted information 
which might be harmful to the college. For example, the issue of community college completers 
is a complex one. Who is a completer? A student who receives a degree or certificate? A student 
who learns a SKill and takes a job? A student who earns enough units to transfer? As a follow- 
up, it would be beneficial to ask some representatives from the conference to indicate their 
perceptions about why these indicators were not chosen as the most important. 



VI. RECOMMENDATIONS FOR FURTHER RESEARCH 

1. The final ten indicators and their measures along with methods used to collect the 
information should be pilot tested in a number of colleges. The results of these pilot tests 
would determine how beneficial these indicators were to the college, how easy they were to 
measure and what benefit the college derived by participating in the process. 

2. The technique of doing small research projects at annual conferences should be explored 
further. The researchers participating in the study felt that the quick turnaround, the large 
captive audience and the small cost were definite advantages which researchers don't often 
encounter. Some controls are lacking as to population, but these seem to be outweighed by the 
advantages. These "mini-research" projects could substantially increase the body of 
knowledge and should be encouraged. 

3. The indicators and measures should further be refined after testing and input from others 
external to the colleges, and a larger sample of students should be included. 
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TABLE I 

NUMBER OF RESPONDENTS FOR ROUND 1 AND ROUND 2 





Round 1 


% 


Round? 




Student 


15 


5 


19 


7 


Faculty 


95 


33 


84 


30 


College Administrator 


83 


29 


80 


29 


District Administrator 


51 


18 


40 


15 


Trustees 


42 


J5 


_53 


J9 


TOTAL 


286 


100% 


276 


100% 
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TABLE II 



ROUND ONE: MEAN AND RANK OF INDICATOR 

(Highest Possible Mean = 4.0) 



Indicator Mean Rank Order 

Faculty Effectiveness 3.82 1 

Positive Faculty/Student Relationships 3.68 2 

Student Satisfaction with Quality of Instruction . 3.62 3 

Positive Reaction of Staff to Students 3.61 4 

Financial Stability 3.54 5 

Student Satisfaction with Services 3.51 6 

Community Perception of the College 3.48 7 

Adequate Facilities to Support College Programs 3.45 8 

Effectiveness of Administrative Staff 3.44 9.5 

Ongoing Program Development 3.44 9.5 

Retention of Students 3.42 1 1 

Ability to Attract New Students 3.41 12 
Positive Management/ Certificated and 

Classified Relationships 3.40 13 
Sufficient Equipment to Support the 

College Programs 3.36 14 

Comprehensiveness of the Curriculum 3.34 15.5 

Effectiveness of Planning 3.34 15.5 
Continuing Staff Development for Faculty/ 

Administrative/ Classified Staff 3.32 17 
Effectiveness of Articulation with Other 

Postsecondary Institutions 3.30 18 

Effectiveness of the District Governing Board 3.28 19 

Overall Student Completion Rate 3.18 20 

Effectiveness of Classified Staff 3.17 21 

Safety within the College Environment 3.08 22 



ERLC 



io 1 5 



Table n 



Indicator Mean Rank Order 

Performance of Community College Students who 

Transfer to a 4-year College or University 3.06 23.5 

Effectiveness of Learning Resource Programs 3.06 23.5 

Enrollment Stability 3.04 25.5 

Job Placement in Major Field of Study 3.04 25.5 

Student Participation Rate * 3.01 27 

Vocational Program Completion Rate 3.00 28 

Effectiveness of Research and Evaluation Activities 2.97 29 
Adequate Participation of All Represented Groups 

in the Community 2.95 30 
Evidence of Articulation Agreements with 

Secondary Schools 2.89 31 

Attractive Physical Environment 2.86 32 
General Community Participation in 

College Activities 2.85 33 



16 

11 



TABLE III 



ROUND 1 

HIGHEST AND LOWEST RANKED INDICATOR FOR EACH RESPONDENT GROUP 



Respondent Group 
Student 

Faculty 

College Administrator 



Trustees 
Overall 



Highest Ranked Indicator 

Student Satisfaction with 
Services (3.87) 

Faculty Effectiveness (3.85) 

Faculty Effectiveness (3.84) 



Faculty Effectiveness (3.88) 
Faculty Effectiveness (3.82) 



Lowest Ranked Indicator 

General Community 
Participation in College 
Activities (2.67> 



Attractive Physical 
Environment (2.81) 



Evidence of Articulation 
Agreements with Other Post- 
secondary Schools (2.90) 



General Community 
Participation; Adequate 
Participation of All 
Represented Groups in the 
Community (2.80) 



Attractive Physical 
Environment (2.71) 



General Community 
Participation in College 
Activities (2.85) 



District Administrator Faculty Effectiveness (3.84) 
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TABLE IV 



ROUND 2 

HIGHEST AND LOWEST RANKED INDICATOR FOR EACH RESPONDENT GROUP 



Respondent Group 
Student 



Highest Ranked Indicator 
Faculty Effectiveness 



Lowest Ra nked Indicator 

Continuing Staff 
Development 



Faculty 



Faculty Effectiveness 



Effective District 
Governing Board 



College Administrator Faculty Effectiveness 



Effectiveness of Articulation 
with Other Postsecondary 
Schools 



District Administrator Faculty Effectiveness 



Effectiveness of Articulation 
with Other Postsecondary 
Schools 



Trustees 



Faculty Effectiveness 



Effectiveness of Articulation 
with Other Postsecondary 
Schools 



Overall 



Faculty Effectiveness 



Effectiveness of Articulation 
with Other Postsecondary 
Schools 
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TABLE V 



OVERALL RANKING OF TOP NINETEEN INDICATORS 
ROUND 1 and ROUND 2 



Round One 

INDICATORS Overall Rank 

1. Faculty Effectiveness 1 

2. Student Satisfaction w/Quallty of 

Instruction 3 

3. Positive Faculty/Student Relationships 2 

4. Financial Stability 5 

5. Effectiveness of Administrative Staff 9 

6. Student Satisfaction with Services 6 

7. Positive Reaction of Staff to Students 4 

8. Community Perception of the College 7 

9 Adequate Facilities to Support College 

Programs 8 

10. Student Retention 11 

11. Effectiveness of Planning 16 

12. Comprehensiveness of Curriculum 15 

13. Positive Management/Certificated & 

Classified Relationships 13 

14. Ongoing Program Development 10 

15. Effectiveness of District Governing Board 19 

16. Continuing Staff Development for Faculty/ 
Administrative/Classified Staff 17 

17. Ability to Attract New Students 12 

18. Sufficient Equipment to Support the 

College Programs 14 

19. Effectiveness of Articulation with 

Other Postsecondary Institutions 18 



Round Two 
Overall Rank 



2 
3 
4 
5 
6 
7 
8 

9 
10 
11 
12 

13 
14 
15 

16 

17 

18 
19 
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TABLE VI 



RANKING OP THE TOP 19 INDICATORS FOR ROUND 2 
BY RESPONDENT CATEGORY 

fTop 10 Round One Rankings In Parenthesis) 



Indicator 


Student Faculty 
N=19 N=§4 


College 
N = 80 


District 
N=40 


Trustee 
N = 53 


Faculty Effectiveness 


1 (7) 


1 (1) 


1 (1) 


1 (1) 


1 (1) 


Student Satisfaction with 
Quality of Instruction 


2 (8) 


2 (2) 


2 (2) 


2 (4) 


4(7) 


Positive Faculty/Student 
Kciaiionsnips 


3(3) 


3(3) 


O f A\ 

3 (4) 


4 (2) 


3 (2) 


r inane lai oiaDiiiiy 


8 (4) 


4 (8) 


4 


3 (9 


2 (4) 


Effectiveness of Administrative 

Qfoff 
OLall 


14 


8 


5 (6J 


5 (6) 


5 (6) 


Student Satisfaction with 
cervices 


4(1) 


7(4) 


6 (9) 


7 (8) 


12 


rOSlUVc KcdCtlOn 01 otaii 

to Students 


7(2) 


5 (5) 


10 (5) 


8 (3) 


8 (5) 


Loinmu niiy rcrcepuon 01 me 
College 


15 


15 


7 (3) 


6(7) 


7 


Adequate Facilities to Support 

r\\ 1 a rf m Dr/ 1 rfromo 

v^uiic^c rTo^rams 


5 


6(7) 


lO MAI 

12 (1UJ 


15 (9) 


9 


oiuuciii rvcLcnuon 


6 (9) 


10 


9 (9 


ii 


11 (10) 


Effectiveness of Planning 


17 


12 


8 


9 


13 


Comprehensiveness of 
Curriculum 


12 


9 


11 


13 


15 (8) 


Positive Management/Certificated 
& Classified Relationships 


18 


14 


13 


14 


10 


Ongoing Program Development 


10 (10) 


13 (6) 


14 


16 


14 (9) 


Effectiveness of District 
Governing Board 


13 (10) 


19. 


17 


10 (10) 


6 (3) 


Continuing Staff Development 
for Faculty /Administrative/ 
Classified Staff 


19 


11 (10) 


16 


12 


18 


Ability to Attract New Students 


9(10) 


17 


15 (7) 


17 


16 


Sufficient Equipment to Support 
the College Programs 


11 


16 (9) 


18 


18 


17 


Effectiveness of Articulation with 
Other Postsecondary Institutions 


16 


18 


19 


19 


19 



These choices were Student Top Ten Round 1, but did not make Inal 19 cut: 
Student Participation Rate (5) 
Overall Student Completion Rate (6) 



ERIC 



TABLE VII 



PERCENT OP RESPONDENTS BT GROUP 
CHOOSING INDICATOR IN TOP 10 







Student 


Faonltv 






Trticfpp 
1 1 UoLCC 


\JVCLciU 






N=19 


N=84 


N=80 


N=40 


N=53 


N = 314 




Indicator 


% 


% 


% 


% 


% 


% 


1. 


Faculty Effectiveness 


94.7 


94.0 


90.0 


92.5 


94.3 


92.4 


o 


Sturlent Satisfaction with 
















Quality of Instruction 


94.7 


88.1 


85.0 


85.0 


64.2 


81.5 


3. 


Positive Faculty/Student 














Relationships 


100.0 


83.3 


68.8 


67.5 


64.2 


73.9 


4. 


Financial Stability 


52.6 


71.4 


83.8 


82.5 


90.6 


79.0 


5. 


Effectiveness of Admini- 
















strative Staff 


47.4 


53.6 


73.8 


75.0 


73.6 


65.0 


a 


Student Satisfaction 
















with Services 


63.2 


60.7 


66.3 


65.0 


43.4 


59.6 


7. 


Positive Reaction of Staff 
















to Students 


68.4 


58.3 


47.5 


47.5 


41.5 


51.6 


8. 


Community Perception of 
















the College 


42.1 


50.0 


67. j 


70.0 


62.3 


58.6 


9. 


Adequate Facilities to 
















Support College Programs 


73.7 


64.3 


48.?; 


37.5 


60.4 


56.1 


10. 


Student Retention 


57.9 


45.2 


51.2 


42.5 


43.3 


48.7 


11. 


Effectiveness of Planning 


26.3 


41.7 


56.2 


40.0 


47.2 


47.8 


12. 


Comprehensiveness of 
















Curriculum 


42.1 


44.0 


45.0 


45.0 


32.1 


40.4 


13. 


Positive Management/ 
Cer llftcated and Classified 
















Relationships 


53 


39.3 


42.5 


40.0 


39.6 


38.5 


14. 


Ongoing Program Development 


63.2 


48.8 


38.8 


32.5 


50.9 


44.9 


15. 


Effectiveness of District 
















Governing Board 


36.8 


45.5 


25.0 


47.5 


66.0 


33.1 


16. 


Continuing Staff Development 
for Faculty/ Administrative/ 
















Classified Staff 


10.5 


47.6 


33.8 


40.0 


30.2 


36.9 


17. 


Ability to Attract New 
















Students 


57.9 


32.1 


38.8 


27.5 


30.2 


35.4 


18. 


Sufficient Equipment to 
















Support the College Program 


36.8 


36.9 


21.2 


27.5 


34.0 


31.8 


19. 


Effectiveness of Articulation 
with Other Postsecondary 
















Institutions 


26.3 


17.9 


17.5 


7.5 


13.2 


17.2 




16 



21 



TABLE VIII 



TOP TEN INDICATORS BY NORTH AND SOUTH COLLEGE DISTRICTS 



Rank 



1 

2 



North (N = 170) 
Faculty Effectiveness 



Positive Faculty/Student 
Relationships 



South fN = 120) 
Faculty Effectiveness 



Positive Faculty/Student 
Relationships 



Student Satisfaction w/Quality 
of Instruction 



Student Satisfaction w/Quality 
of Instruction 



Effectiveness of Administrative Effectiveness of Administrative 
Staff staff 



Adequate Facilities to Support Student Satisfaction with 
College Programs Services 



Financial Stability 



Adequate Facilities to Support 
College Programs 



Student Satisfaction with 
with Services 



Financial Stability 



8 



Community Perception of 
the College 



Effectiveness of Planning 



9 
10 



Positive Reaction of Staff 
to Student 

Effectiveness of Planning 



Positive Reaction of Staff 
to Students 

Effectiveness of District 
Governing Board 
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TABLE IX 



RANKING BY THE AGE CATEGORY OF THE RESPONDENT 

Indicator Under 40 41-50 51-60 Over 61 

1. Faculty EiTectiveness 1111 

2, Student Satisfaction w/Quality 

of Instruction 2 2 3 2 

a Positive Faculty/Student 3 3 2 3 

Relationships 

4. Financial Stability 4 4 4 4 

5. Effectiveness of 

Administrative StafT 7 5 5 5 

& Student Satisfaction 

with Services 5 6 8 6 

7. Positive Reaction of Staff 

to Students 8 7 6 11 

8. Community Perception 

of the College 10 8 10 8 

9. Adequate Facilities to 

Support College Programs 11 11 7 7 

10. Student Retention 6 9 9 12 

11. Effectiveness of Planning 9 10 13 13 

12. Comprehensiveness of 

Curriculum 12 12 12 18 

13. Positive Management/ 
Certificated & Classified 

Relationships 15 15 11 9 

14. Ongoing Program 

Development 13 13 16 15 

15. Effectiveness of District 

Governing Board 18 17 14 10 

16. Continuing Staff Development 
for Faculty/Administrative/ 

Classified Staff 16 14 15 14 

17. Ability to Attract New 

Students 14 15 17 17 

18. Sufficient Equipment to 

Support the College Programs 16 18 18 16 

19. Effectiveness of Articulation 
with Other Postsecondary 

Institutions 19 19 19 19 
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TABLE X 



ROUND 2 RESULTS FOR RESPONDENTS WHO WORKED 
WITHIN A SINGLE OR WLTI-CAMPUS ENVIRONMENT 



Single Campus Multi-Campus 

Indicator Rank Rank 

1. Faculty Effectiveness 1 1 

% Student Satisfaction w/Quality of 

Instruction 3 3 

3. Positive Faculty/Student Relationships 2 2 

4. Financial Stability 7 7 

5. Effectiveness of Administrative Staff 4 4 
& Student Satisfaction with Services 5 6 

7. Positive Reaction of Staff to Students 10 8 

8. Community Perception of the College 8 9 

9. Adequate Facilities to Support College 

Programs 6 5 

10. Student Retention 12 16 

11. Effectiveness of Planning 9 10 

12. Comprehensiveness of Curriculum 14 17 

13. Positive Management/Certificated and 

Classified Relationships 15 13 

14. Ongoing Program Development 18 18 

15. Effectiveness of District Governing Board 13 11 

16. Continuing Staff Development for Faculty/ 
Administrative/Classified Staff 11 12 

17. Ability to Attract New Students 16 15 

18. Sufficient Equipment to Support the 

College Programs 19 19 

19. Effectiveness of Articulation with Other 

Postsecondaiy Institutions 17 14 



9 
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APPENDIX A 

ROUND ONE - QUESTIONNAIRE 



November 15, 1987 

Dear Conference Participant: 

We would like to ask you to take five minutes to participate In an Important research activity 
to Identify indicators of successful community colleges. 

The attached questionnaire allows you to express your professional judgment about how im- 
portant each stated indicator is in determining successful community colleges. Please complete 
the questionnaire and turn it in at the end of the session. 

Your responses will be tabulated and presented at the luncheon on Saturday. At that time, 
you will also receive a second round questionnaire which will allow you to further refine your 
choices. 

We feel this is a unique opportunity for all constituencies to share their views, 
Thanks for your help. 



The CACC Research 
Commission 
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DEMOGRAPiilC INFORMATION 



To allow us to see if different opinions exist within specific groups, please complete the following 
demographic information: 



1. IDENTIFICATION fchcck one) 



} Student 
"** adent Trustee 



I I Faculty 

I I District Aministrator 

I 1 College Administrator 

I 1 Trustee 

I I Classified Staff 

I I State Agency Employee 

I I Other (specify) 

2. IF FACULTY". YOUR PRIMARY ROLE fcheck one! 

1 1 Instructor 
I I Counselor 
Librarian 

Department/Division Chair 
Other (specify) 



4. YOUR AGE (check one) 



j Under 20 years 



I , I 20 - 25 years 

I I 26 -30 years 

I I 31 -40 years 

I I 41-50 years 

I , I 51 - 60 years 

I I 61 - 70 years 

I ! 71 and.above 



5. LOCATION OF DISTRICT (chcckonc) 
I I Bakersfield and South 



I I North of Bakersfield 



6. TYPE OF ORGANIZATION 
(check one) 



Multi-Campus District 



j Single Campus District 



3. IF ADMINISTRATOR, FUNCTION CLOSEST 
TC 7£>IIE3 (check one) 



I l President/ Superintendent 

I I Business Services Administration 

I I Instructional Administration 

I I Student Services Administration 

I I Research Administration 

I I Grants Administration 

I I Personnel Administration 

Other (specify) 



IF YOU WORK ON A COLLEGE 
CAMPUS. FALL. 1987 HFj\D - 
COUNT (check one) 



CZ) 



0 - 4,999 
5,000 - 9.999 



I I 10,000 - 15,999 
I I 16,000 - 19.999 
IZD 20,000-29.999 
30,000 and over 
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INDICATORS OF SUCCESSFUL COMMUNITY COLLEGES 



INDICATOR 

1 . Community Perception of the College 

2. Adequate Facilities to Support College 
Programs 

3. Positive Faculty/Student Relationships 

4. Effectiveness of Articulation with Other 
Postsecondary Institutions 

5. Ability to Attract New Students 

6. General Community Participation in 
College Activities 

7. Comprehensiveness of the Curriculum 

8. Effectiveness of Administrative Staff 

9. Continuing Staff Development for 
Faculty/Administrative/Classified Staff 

10. Performance of Community College Students 
Who Transfer to a 4-Year College or University 

1 1 . Enrollment Stability 

12. Job Placement m Major Field of Study 



How Important is This as an Indicator of Success? 



Extremely Very Somewhat Not Very 

Important Important Important Important 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 



ERLC 



27 



28 



INDICATORS OF SUCCESSFUL COMMUNITY COLLEGES 



INDICATOR 

1 3. Effectiveness of Learning Resource Programs 

14. Evidence of Articulation Agreements with 
Secondary Schools 

15. Student Satisfaction with Services 

16. Effectiveness of Classified Staff 

1 7. Positive Reaction of Staff to Students 

18. Effectiveness of Research and Evaluation 
Activities 

19. Vocational Program Completion Rate 

20. Student Participation Rate 

2 1 . Effectiveness of the District Governing Board 

22. Attractive Physical Environment 

23. Financial Stability4 

24. Ongoing Program Development 

29 



How Important is This as an Indicator of Success? 



Extremely Very Somewhat Not Very 

Important Important Important Important 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

3 2 l 

4 3 2 1 
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INDICATORS OF SUCCESSFUL COMMUNITY COLLEGES 



INDICATOR 

25. Retention of Students 

26. Overall Student Completion Rate 

27. Effectiveness of Planning 

28. Positive Management/Certificated and 
Classified Relationships 

29. Adequate Participation of All Represented 
Groups in the Community 

30. Student Satisfaction with Quality of 
Instruction 

3 1 . Faculty Effectiveness 

32. Sufficient Equipment to Support the 
College Programs 

33. Safety Within the College Environment 

34. Other (specify) 



How Important is This as an Indicator of Success? 



Extremely Very Somewhat Not Very 

Important Important Important Important 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 

4 3 2 1 
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APPENDIX B 

ROUND TWO - QUESTIONNAIRE 

CALIFORNIA ASSOCIATION OF COMMUNITY COLLEGES COMMISSION ON RESEARCH 
INDICATORS OF SUCCESSFUL COMMUNITY COLLEGES 

In the opening session of the Convention, you were asked to indicate the importance of 33 
items in measuring successful community colleges. The results of this survey presented below show 
the most important indicators along with their average score. 

Today, we want you to refine your choices Ly identifying the 10 most important indicators 
among the 1 9 and then by ranking them from 1-10. Number one would be your first ranked indicator. 



RANK 



INDICATOR 



AVERAGE SCORE 
ROUND 1 



9 
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1. Faculty Effectiveness 3.82 

2. Positive Faculty/Student Relationships 3.68 

3. Student Satisfaction with Quality of Instruction 3.62 

4. Positive Reaction of Staff to Students 3.61 

5. Financial Stability 3.54 

6. Student Satisfaction with Services 3.51 

7. Community Perception of the College 3.48 

8. Adequate Facilities to Support College Programs 3.45 

9. Effectiveness of Administrative Staff 3.44 

10. Ongoing Program Development 3.44 

1 1 . Retention of Students 3.42 

12. Ability to Attract New Students 3.4 1 

13. Positive Management/Certificated and Classified Relationships 3.40 

14. Sufficient Equipment to Support the College Programs 3.36 

15. Comprehensiveness of the Curriculum 3.34 

16. Effectiveness of Planning 3.34 

17. Continuing Staff Development for Faculty/ 
Administrative/Classified Staff 3.32 

18. Effectiveness of Articulation with Other Postsecondaiy Institutions 3.30 

19. Effectiveness of District Governing Board 3.28 
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APPENDIX C 

MEASURES QUESTIONNAIRE 

INDICATORS OF SUCCESSFUL COMMUNITY COLLEGE PROJECT 



#1 FACULTY EFFECTIVENESS 

Survey of faculty satisfaction. 

Ability of faculty to attract students to classes. 

Retention rates from 1st census to 2nd census to end of term. 

Awards received by faculty. 

Classified and administrative staff perception of faculty. 

Proportion of hours taught by full-time faculty. 

Proportion of Vocational Education faculty who have been employed in the field they are 
teaching. 



Student perceptions of faculty effectiveness. 

Results of periodic faculty evaluation by peers, students, and supervising administrator. 

Proportion of faculty involved in research in content/ curriculum development in their 
own field. 

Degree to which students meet stated course objectives. 



#2 STUDENT SATISFACTION WITH QUALITY OF INSTRUCTION 

Currently enrolled students* perceptions of instructors from required, standardized 
evaluation form. 



— Students* self-reports upon exiting a course. 

Students* self-reports gathered after leaving (with a time interval, e.g.: one semester 
_ later). 



Classroom retention (passing grades divided by initial enrollment). 
Student re-enrollment with the same instructor. 
Consensus results from student focus groups. 
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#3 POSITIVE FACULTY/STUDENT RELATIONSHIPS 

Amount of out-of-class time faculty spend with students. 

Students* perceptions of faculty as positive role models. 

Faculty perception of students. 

Reasons why students come to faculty outside of class. 

Did student feel recognized as an individual in this class? 

Did student look forward to coming to this class? 

How often does faculty inquire about student needs (vs. goals)? 

Does instructor give adequate feedback regarding student progress? 

Is student aware of a grievance system? 

When dissatisfied, what did student do (when did they go to, etc.)? 

Student retention rates by instructor (with control of as many variables as possible 
such as course type, hour presented, etc.)? 



#4 FINANCIAL VIABILITY * 

Amount and number of sources of external funding generated in proportion to general 
fund. 



Proportion of expenditures used from reserves for college operation. 

Percent of general reserves to annual budget. 

Revenue per unit of workload. 

Expenditures per unit of workload. 

Diversity of revenue sources. 

Positive auditor's report. 

Level of college endowment, foundation resources. 

Number of profit-making ventures, including lease of land. 

Extent of financial planning. 

*A11 measures should be assessed over time (5 years). 
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#5 EFFECTIVENESS OF ADMINISTRATIVE STAFF 

Leadership role In the community college system as shown by number of publications, 
presentations and leadership roles. 



Ability to motivate college community. 
Fiscal solvency. 
Staff morale. 

Amount of positive vs. negative press coverage. 
Effectiveness in carrying out job descriptions. 
Verbal /written presentation skills. 

Evidence of key decisions being linked to planning process. 
The extent faculty/staff is involved in decision-making process. 



#6 STUDENT SATISFACTION WITH SERVICES (what a college provides outside 

classroom) 

Satisfaction survey while in attendance (a student at the college). 

Satisfaction survey after leaving (after separation from college). 

Satisfaction survey at time of service delivery. 

Interview students about their experience. 

Student involvement in services as a percentage of enrollment. 

Unsolicited letters to college, paper. 

Amount of time it takes to provide a service. 

Number of referrals to services by other students. 

Increase in number of student-body cards sold at bookstore. 
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#7 POSITIVE REACTION OF STAFF TO STUDENTS 

Number of complaints received about services. 

Number of compliments received about services. 

Perceptions of students of helpfulness of college staff. 

Staff satisfaction with job environment. 

Retention information. 

Frequency with which students make use of faculty office hours. 

Diversity of students who show up for office hour opportunities. 

Instructor attitudes of student learning. 

Appreciation/awareness of learning modalities as measured by claesroom assessment 
and inclusion of modalities in lesson plans. 



Percentage of students that instructor likes. 



#8 COMMUNITY PERCEPTION OF THE COLLEGE 

Number of students attending who were referred by a former student or employee. 

Amount and quality of favorable newspaper media coverage. 

Solicited community attitudes using various methods (surveys, telephone contacts, 

focused interviews) of various groups such as employers, high school counselors, 
underrepresented populations. Formal needs assessment, including perceived atti- 
tudes, is the most comprehensive method. 

Percentage of local high school graduates attending college. 

Participation rate of population of the District in college activities. 

Number of requests for service from business, industiy and other agencies. 
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#9 - ADEQ UATE FACILITIES TO SUPPORT COLLEGE PROGRAMS 

Number of square feet In permanent vs. temporary facilities 

Dollars spent on maintenance per ASF. 

Dollar amount of emergency replacement of facilities. 

Amount of non-instructional space able to be scheduled for other activities. 

Ratio of instructional to non-instructional space. 

ASF/WSCH by discipline area. 

Number of hours of room availability compared to number of hour? used. 

Proportion of seats filled when in use. 

, Adequate mix of classroom sizes from Seminar to Lecture Hall. 

Flexibility /Adaptability of floor space to meet changing needs. 

Adequacy of equipment to meet current instructional needs. 



#10 RETENTION OF STUDENTS 

(Note: Each of the measures listed below is to be cross tabulated with various student variables, 
e.g.: by basic skill level or by student goal.) 

Total units completed divided by total units attempted at 1st census (unit earned rate). 

Re-enrollment rates (over time). 

Chang es in re-enrollments over time within the institution. 

ghanges In unit earned rate over time within the institution. 
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